---
title: "How to stream Postgres to GCP Pub/Sub"
sidebarTitle: "GCP Pub/Sub"
description: "Integrate Postgres with Google Cloud services through Pub/Sub. Stream database changes for event processing, Cloud Functions, and more."
---

This guide shows you how to set up Postgres change data capture (CDC) and stream changes to Google Cloud Pub/Sub using Sequin.

With Postgres data streaming to Pub/Sub, you can trigger workflows, keep services in sync, [build audit logs](/how-to/create-audit-logs), [maintain caches](/how-to/maintain-caches), and more.

By the end of this how-to, you'll have database changes flowing to a Pub/Sub topic.

<Tip>
  This is the how-to guide for streaming Postgres to GCP Pub/Sub. See the [quickstart](/quickstart/gcp-pubsub) for a step-by-step walkthrough or the [reference](/reference/sinks/gcp-pubsub) for details on all configuration options.
</Tip>

## Prerequisites

If you're self-hosting Sequin, you'll need:

1. [Sequin installed](/running-sequin)
2. [A database connected](/connect-postgres)
3. A Google Cloud project


3. A Google Cloud project

## Basic setup

### Enable Pub/Sub

In the Google Cloud console, navigate to [Pub/Sub](https://console.cloud.google.com/cloudpubsub) and ensure it's enabled for your project.

### Create a Pub/Sub topic

1. Navigate to the [Pub/Sub Topics page](https://console.cloud.google.com/cloudpubsub/topic/list) in your Google Cloud Console
2. Click "CREATE TOPIC"
3. Enter a topic ID (e.g., "postgres-changes")
4. (Optional) Enable "message retention" if you want to be able to replay messages
4. Click "CREATE"

Make note of your:
- Google Cloud project ID (e.g., "my-project-123")
- Pub/Sub topic ID (e.g., "postgres-changes")

### Create a service account

1. Navigate to the [IAM & Admin > Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts) page
2. Select a project (if one is not already selected)
3. Click "CREATE SERVICE ACCOUNT"
4. Enter a name (e.g., "sequin-publisher")
4. Click "CREATE AND CONTINUE"
5. Add the following two roles:
   - "Pub/Sub Publisher" (`roles/pubsub.publisher`)
   - "Pub/Sub Viewer" (`roles/pubsub.viewer`)
6. Click "CONTINUE"
6. Click "DONE"
7. Click on the newly created service account
8. Go to the "KEYS" tab
9. Click "ADD KEY" > "Create new key"
10. Choose "JSON" format
11. Click "CREATE"

The JSON key file will be downloaded to your computer. Keep this safe - you'll need it when configuring the Pub/Sub sink in Sequin.

## Create Pub/Sub sink

In Sequin, navigate to the "Sinks" tab, click "Create Sink", and select "Pub/Sub Sink".

### Configure the source

<Steps>
  <Step title="Select source tables">
    Under "Source", select the schemas and tables you want to stream data from.
  </Step>

  <Step title="Add filters (optional)">
    Add [filters](/reference/filters) to the sink to control which database changes are sent to your Pub/Sub topic.
  </Step>

  <Step title="Specify backfill">
    You can optionally indicate if you want Pub/Sub to receive a [backfill](reference/backfills) of all or a portion of the table's existing data. Backfills are useful if you want to use Pub/Sub to process historical data.

    You can backfill at any time. If you don't want to backfill, toggle "Backfill" off.
  </Step>

  <Step title="Configure message grouping">
    Under "Message grouping", you can control how messages are grouped for ordering:

    - **Enable message grouping** (default): Messages are grouped by primary key to ensure ordered delivery within each group. This is recommended if you plan to use Pub/Sub subscriptions with "message ordering" enabled.
    - **Disable message grouping**: Messages are not grouped, allowing for higher throughput but without ordering guarantees.

    If you enable message grouping, you can also specify custom grouping columns at the table level to group by fields other than the primary key (e.g., group orders by `customer_id`).

    When message grouping is enabled and you use a Pub/Sub subscription with "message ordering" enabled, your system will receive events for each group in the [same order they occurred in Postgres](/reference/sinks/gcp-pubsub#grouping-and-ordering).
  </Step>
</Steps>

### Configure Pub/Sub

<Steps>
  <Step title="Enter Pub/Sub details">
    Fill in your Pub/Sub connection details:

    - **Project ID** (required): Your Google Cloud project ID (e.g., "my-project-123")
      - You can find this on the home page of the Google Cloud Console
    - **Topic ID** (required): The Pub/Sub topic ID (e.g., "postgres-changes")
    - **Service Account Credentials** (required): Paste the entire contents of your service account JSON key file
  </Step>

  <Step title="Confirm connection">
    Click "Test Connection" to verify that Sequin can connect to your Pub/Sub topic.
  </Step>

  <Step title="Create the sink">
    Give your sink a name, then click "Create Pub/Sub Sink".
  </Step>
</Steps>

## Verify & debug

To verify that your Pub/Sub sink is working:

1. Make some changes in your source table
2. Verify that the count of messages for your sink increases in the Sequin web console
3. In the Google Cloud Console:
   - Navigate to your Pub/Sub topic
   - Click on the "MESSAGES" tab
   - Under "Step 2", select a Cloud Pub/Sub subscription
   - Click "PULL" to see messages in the topic

If messages don't seem to be flowing:

1. Click the "Messages" tab to view the state of messages for your sink
2. Click any failed message
3. Check the delivery logs for error details, including any Pub/Sub API errors

Common issues to check:
- Service account credentials are valid and not expired
- Service account has the "Pub/Sub Publisher" role
- Project ID and Topic ID match exactly
- Topic exists in the specified project

## Next steps

- **Setup a processor**

   Now that your Postgres data is flowing into Pub/Sub, you can setup a subscriber to read from the topic and process the data. See the [Pub/Sub documentation](https://cloud.google.com/pubsub/docs/pull) for details on consuming messages.

   Refer to the [Pub/Sub sink reference](/reference/sinks/gcp-pubsub) for the shape of messages that Sequin will publish.

- **Deploy your implementation**

   When you're ready to deploy your implementation, see "[How to deploy to production](/how-to/deploy-to-production)".

- **Advanced configuration**

   For more about how Pub/Sub sinks work, see the [Pub/Sub sink reference](/reference/sinks/gcp-pubsub).